Skip to content

bun:test: do not call then() on what a callback with a done parameter returns - #47730

Open
robobun wants to merge 3 commits into
mainfrom
robobun/871123bb/done-callback-no-then
Open

robobun wants to merge 3 commits into
mainfrom
robobun/871123bb/done-callback-no-then

Conversation

@robobun

@robobun robobun commented Nov 2, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

  • Since bun test: Vitest compatibility #44833 bun test calls then() on a thenable that a test or a hook returns, also when the callback takes done. supertest's done => request(app).get("/").expect(200, done) then fails: .end() was called twice. 1.4.4 passes it.
  • Cause: run_test_callback (src/runtime/test_runner/bun_test.rs:1392) gives every returned value to promise_of_thenable. It does not read cfg_done_parameter.

Fix

  • With a done parameter the runner skips promise_of_thenable: no then() call on the returned value, as on 1.4.4. A returned promise is still awaited by its state.
  • Correct: such a callback ends through done(). Jest 30.5.2 and node --test 26.3.0 call no then() there.
  • Verified: test/js/bun/test/vitest-flavor.test.ts (2 new tests, both fail on canary 3f3a9ff87). Also test/js/bun/test/, node-test.test.ts. Self-reviewed: 4 concerns raised, 4 addressed.

Background

  • A thenable is an object with then() that is not a promise. A supertest request is one: end() sends it, and then() calls end().
  • done is the parameter of a callback from bun:test or @jest/globals. With vitest it is the test context, and nothing changes.
  • Weighed: to fail the test, as Jest does (Test functions cannot both take a 'done' callback and return something). That fails suites that pass on 1.4.4.

Downsides

  • It reverses a behaviour of main that is in no release. A done test whose returned thenable rejects or never settles fails on main. With this PR it passes, as on 1.4.4.
  • Speed and size: none found. 0 added instructions on 4 measured paths, size text unchanged.
Notes

What changes

Each row was run on 1.4.4 (38afee6), on the canary of main (1.4.4-canary.1+3f3a9ff87, with #44833) and on a release build of this branch. Every callback takes done.

Callback 1.4.4 main this PR
done => request(app).get("/").expect(200, done) (supertest 6.3.3) pass fail: .end() was called twice. superagent warns that the request was sent twice pass
done => request(app).get("/").expect(200).end(done) pass fail, the same error pass
The same request on a route that answers 404 fail: expected 200 "OK", got 404 "Not Found" fail: .end() was called twice fail: expected 200 "OK", got 404 "Not Found"
done(); return a thenable that rejects pass, 0 then() calls fail, 1 call pass, 0 calls
done(); return a thenable that never settles pass timeout pass
Returns a thenable, never calls done() timeout timeout timeout
Returns a thenable, calls done(error) later fail with the error fail with the error fail with the error
Returns a promise, native or subclass awaited by its state, and done() the same the same
  • On main the late callback of the failed supertest request also fails the test that runs next.
  • Rows 4 and 5 are the two results that go from fail to pass against main. Main waits for both done() and the thenable. This PR waits for done() only. That is the cost of the rule: a rejection of a thenable that a done test returns is not seen, as on 1.4.4.
  • A callback without done does not change. It still waits on a returned thenable (Bun test runner passes supertest request, where Jest fails #27945).

Other runners

Run with a returned thenable and a returned Promise subclass in a done test. Each makes 0 then() calls.

  • Jest 30.5.2 fails the test: Test functions cannot both take a 'done' callback and return something. Either use a 'done' callback, or return a promise.
  • node --test 26.3.0 ignores the thenable and passes. It fails a returned promise: passed a callback but also returned a Promise.
  • Vitest has no done callback.

Related

Measurements

Release builds of main (3f3a9ff) and of this branch from one build directory, linux-x64, JIT off.

  • Instructions inside run_test_callback, callees included, per call: undefined 982 to 982, plain object 1380 to 1380, pending native promise 5350 to 5350, done => done() 3242 to 3242. Method: a gdb single-step from the entry of the function to its return, at the fourth and the fifth call.
  • size text: 140287599 to 140287599. run_test_callback 2426 B to 2428 B.
  • perf and valgrind are not in the container.

Tests

  • The new test runs a supertest-shaped thenable from a hook, a test and test.each, a thenable that rejects, one that never settles, a done() that is never called and a late done(error). It passes on 1.4.4 and fails on main.
  • Debug build: vitest-flavor.test.ts, done-async.test.ts, test-test.test.ts, bun_test.test.ts, concurrent.test.ts, jest-hooks.test.ts, test-error-code-done-callback.test.ts and node-test.test.ts: 306 pass. The new test is clean with BUN_JSC_validateExceptionChecks=1.
  • Release build: test/js/bun/test/ 4188 pass, 1 fail. The failure is "error snapshots" in snapshot.test.ts, which expects a TTY and fails on the release build of main too.

[auto-merge] gate passed · iteration 1 · 3 files touched

fails on main (without fix)
ASAN without fix: 2 FAILED
$ BUN_DEBUG_QUIET_LOGS=1 bun scripts/build.ts --profile=debug --quiet test "--reporter=junit" "--reporter-outfile=/tmp/pr_gate.xml" test/js/bun/test/vitest-flavor.test.ts
bun test v1.4.4 (367d939d9)

test/js/bun/test/vitest-flavor.test.ts:
(pass) test context > TestRunner.getCurrentTest() in a concurrent test [248.75ms]
(pass) test context > a parameter is the context, never a done callback [354.88ms]
(pass) test context > is the same object in the hooks, the test, its retries and its repeats [288.01ms]
(pass) test context > TestRunner.getCurrentTest() is the task of the running test [309.97ms]
(pass) test context > is callable, and calling it like done() throws a deprecation error [328.50ms]
(pass) test context > task [297.98ms]
(pass) test context > onTestFinished and onTestFailed of the context, in concurrent tests [264.70ms]
(pass) test context > onTestFinished and onTestFailed run after the hooks, last registered first [296.37ms]
(pass) test context > task.result is updated as the test runs [332.61ms]
(pass) test context > skip() [340.30ms]
(pass) test context > onTestFinished and onTestFailed outside of a test [273.74ms]
(pass) test context > annotate() record
... (truncated)

release without fix: 170 FAILED
bun test v1.4.3-canary.1 (367d939d9)

test/js/bun/test/vitest-flavor.test.ts:
168 |             console.log(name, "after an await", TestRunner.getCurrentTest());
169 |           });
170 |         }
171 |       `,
172 |     });
173 |     expect({ log: log.sort(), exitCode }).toEqual({
                                                ^
error: expect(received).toEqual(expected)

  {
-   "exitCode": 0,
-   "log": [
-     "first after an await undefined",
-     "first true",
-     "second after an await undefined",
-     "second true",
-   ],
+   "exitCode": 1,
+   "log": [],
  }

- Expected  - 7
+ Received  + 2

      at <anonymous> (/workspace/bun/test/js/bun/test/vitest-flavor.test.ts:173:43)
251 |           });
252 |         });
253 |         test.concurrent("concurrent", ({ task }) => { console.log(task.concurrent); });
254 |       `,
255 |     });
256 |     expect({ log, exitCode }).toEqual({
                                    ^
error: expect(received).toEqual(expected)

  {
-   "exitCode": 0,
+   "exitCode": 1,
    "log": [
-     "{"id":"string","type":"test","name":"top","fullTestName":"top","fullName":"dir/a.test.js > top","mode":"run","file":"dir/a.test.js","ti
... (truncated)
passes on PR (with fix)
ASAN with fix: all passed
$ BUN_DEBUG_QUIET_LOGS=1 bun scripts/build.ts --profile=debug --quiet test "--reporter=junit" "--reporter-outfile=/tmp/pr_gate.xml" test/js/bun/test/vitest-flavor.test.ts
bun test v1.4.4 (367d939d9)

test/js/bun/test/vitest-flavor.test.ts:
(pass) test context > a parameter is the context, never a done callback [390.42ms]
(pass) test context > TestRunner.getCurrentTest() is the task of the running test [360.15ms]
(pass) test context > is the same object in the hooks, the test, its retries and its repeats [360.58ms]
(pass) test context > is callable, and calling it like done() throws a deprecation error [388.36ms]
(pass) test context > TestRunner.getCurrentTest() in a concurrent test [390.80ms]
(pass) test context > task [236.70ms]
(pass) test context > onTestFinished and onTestFailed run after the hooks, last registered first [278.24ms]
(pass) test context > task.result is updated as the test runs [311.27ms]
(pass) test context > onTestFinished and onTestFailed of the context, in concurrent tests [283.80ms]
(pass) test context > skip() [318.31ms]
(pass) test context > onTestFinished and onTestFailed outside of a test [297.88ms]
(pass) test context > annotate() record
... (truncated)

release with fix: all passed
$ bun scripts/build.ts --profile=release
[configured] bun-profile → bun (stripped) in 819ms (unchanged)
ninja: Entering directory `/workspace/bun/build/release'
[1/352] gen generated_host_exports.rs
generated_host_exports.rs: 119 exports (lazy=4, generic=115, rust=0); 288 extern-C blocks audited
[2/351] gen the table of built-ins
builtins table: operands without uses: none
builtins table: 2101 host entries have a result that is not described; a program in which a value may be anything holds what any host class needs: 0 modules, 15 functions
[3/82] cxx obj/codegen/GeneratedByteStream.cpp.o
[4/82] cxx obj/codegen/GeneratedFileSink.cpp.o
[5/82] cxx obj/codegen/GeneratedFetchTasklet.cpp.o
[6/82] cxx obj/codegen/GeneratedFetch.cpp.o
[7/82] cxx obj/codegen/GeneratedFormData.cpp.o
[8/82] cxx obj/codegen/GeneratedFetchSession.cpp.o
[9/82] cxx obj/codegen/GeneratedCrypto.cpp.o
[10/82] cxx obj/codegen/GeneratedBlob.cpp.o
[11/82] cxx obj/codegen/GeneratedTimer.cpp.o
[12/82] cxx obj/codegen/GeneratedTestContext.cpp.o
[13/82] cxx obj/codegen/GeneratedValkey.cpp.o
[14/82] cxx obj/codegen/GeneratedProxyTesting.cpp.o
[15/82] cxx obj/codegen/GeneratedPrompt.cpp.o
[16/82] cxx obj/c
... (truncated)
diff hotspot
docs/test/lifecycle.mdx                |  2 +-
 src/runtime/test_runner/bun_test.rs    |  3 ++-
 test/js/bun/test/vitest-flavor.test.ts | 44 ++++++++++++++++++++++++++++++++++
 3 files changed, 47 insertions(+), 2 deletions(-)

gate history · 1 passed · 0 rejected · iteration 1

evidence per changed file
file                                    reads  edits  tests
docs/test/lifecycle.mdx                     1      1    108
src/runtime/test_runner/bun_test.rs        13     19    119
test/js/bun/test/vitest-flavor.test.ts      8     11    107

… returns

A callback that takes `done` ends through `done()`. The runner no longer
calls `then()` on an object that such a callback also returns.

Since #44833 the runner followed a thenable that a `done` callback
returned. That sent a supertest request twice:
`done => request(app).get("/").expect(200, done)` failed with
".end() was called twice". Bun 1.4.4, Jest 30.5.2 and `node --test` 26.3.0
do not call `then()` there.
@coderabbitai

coderabbitai Bot commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Recent review info
Run configuration
Commits

Reviewing files that changed from the base of the PR and between 3f3a9ff and fa96028.

Files selected for processing: 3
  • docs/test/lifecycle.mdx
  • src/runtime/test_runner/bun_test.rs
  • test/js/bun/test/vitest-flavor.test.ts

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 1 remain after this review.

Walkthrough

The test runner no longer processes returned thenables when a callback accepts a done parameter. Regression tests cover hooks and test callbacks, and the lifecycle documentation describes this behavior.

Changes

Done callback thenable handling

Layer / File(s) Summary
Done callback behavior
src/runtime/test_runner/bun_test.rs, test/js/bun/test/vitest-flavor.test.ts, docs/test/lifecycle.mdx
Callbacks with a done parameter use the callback directly instead of converting its return value to a promise. Tests cover hooks, ordinary tests, test.each, rejection, non-settlement, timeout, and done(error). The lifecycle documentation states that Bun does not call then() on the returned object.

Suggested reviewers: jarred-sumner, alii

Merge Risk: Low · up to fa960

The done-callback change has no established runtime regression. The two explicit test timeouts need a small cleanup or a justified exception; this is a bounded maintenance concern.

Pre-merge checks: Passed 4
Check name Status Explanation
Title check Passed The title clearly and concisely describes the main change: Bun no longer calls then() on values returned by callbacks that accept done.
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Description check Passed The description follows the repository template with both required sections present: "What does this PR do?" is covered by the Problem, Fix, and Background subsections, and "How did you verify your co…
Full details: Description check

Explanation

The description follows the repository template with both required sections present: "What does this PR do?" is covered by the Problem, Fix, and Background subsections, and "How did you verify your code works?" is covered by the Verified line, the Tests section, and the Measurements section. It explains the cause (run_test_callback passes every returned value to promise_of_thenable without checking cfg_done_parameter), the behavior change for done callbacks, the tradeoff (a rejecting or never-settling thenable returned from a done test now passes, as on 1.4.4), and the new regression tests in test/js/bun/test/vitest-flavor.test.ts, which fail on canary and pass with the fix. The docs change in docs/test/lifecycle.mdx is also mentioned. The description is detailed and specific. The template headings are not reproduced verbatim, but the content maps onto both required sections.

  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

CodeRabbit Verify

  • Run CodeRabbit Verify to test the changes in this pull request.

@robobun

robobun commented Nov 2, 2026 •

Copy link
Copy Markdown
Collaborator Author
Updated 5:40 AM PT - Nov 2nd, 2026

❌ @robobun, your commit fa96028 has 41 failures in Build #136842 (All Failures):


🧪   To try this PR locally:

bunx bun-pr 47730

That installs a local version of the PR into your bun-47730 executable, so you can run:

bun-47730 --bun

@github-actions github-actions Bot added the claude label Nov 2, 2026
@robobun

robobun commented Nov 2, 2026 •

Copy link
Copy Markdown
Collaborator Author

Reproduced on the canary of main, 1.4.4-canary.1+3f3a9ff87, with supertest 6.3.3:

  • test("x", done => request(app).get("/").expect(200, done)) fails with .end() was called twice. This is not supported in superagent. superagent warns that the request was sent twice.
  • The same test passes on 1.4.4.

The new test in test/js/bun/test/vitest-flavor.test.ts fails on that canary and passes on a debug build of this branch.

CI (build 137060): the red files are test/aot/* and test/bundler/compile-aot-size.test.ts. They are red on main too. No test of this PR is among them.

PR: #47730

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing blocking. The comments below are optional suggestions. There is no need to push a fix for them before merging.

Beyond the inline nit, I also checked what happens when a done callback returns a callable thenable and calls done() synchronously: the raw value now reaches adopt_teardown, but that function returns early unless calling.is_vitest(), and a done parameter only exists for Calling::Jest, so no teardown is registered. The dropped rejection of a returned thenable under done is the intended trade-off the description states, not a defect in the gate.

Extended reasoning...

The change gates the promise_of_thenable step in BunTest::run_test_callback on cfg_done_parameter in src/runtime/test_runner/bun_test.rs, adds one doc sentence to docs/test/lifecycle.mdx, and adds a spawned-fixture test matrix for bun:test and @ jest/globals in test/js/bun/test/vitest-flavor.test.ts; it touches no security-sensitive surface. One inline nit about a 10 ms timeout in the fixture was posted; the two candidate correctness issues were traced in the code and do not apply.

Comment thread test/js/bun/test/vitest-flavor.test.ts Resolved

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed the updated PR and didn't find any bugs; the follow-up commit addressed the timeout concern from my earlier inline comment. A human look is still worthwhile only for the one policy choice this makes: a thenable returned by a done callback is now ignored (its rejection is never seen), which restores 1.4.4 behaviour rather than failing the test the way Jest does.

What was reviewed:

  • run_test_callback in src/runtime/test_runner/bun_test.rs: cfg_done_parameter is Calling::Jest { done: true } from Execution.rs, so vitest callbacks (where the parameter is the context) are unaffected, and native promises returned with done still go through the existing as_promise() state tracking.
  • The fixture's log assertion (exactly three sent lines, no then() lines) is the load-bearing check that no then() is called from a hook, a test, or test.each; the 10 ms timeout is only on the case expected to fail, and the timeout message is indented so it does not leak into errors.
  • Doc sentence in docs/test/lifecycle.mdx matches the implemented behaviour.
Extended reasoning...

The change is a one-line conditional in src/runtime/test_runner/bun_test.rs that skips promise_of_thenable when the callback declares a done parameter, plus a subprocess test in test/js/bun/test/vitest-flavor.test.ts covering bun:test and @ jest/globals and a one-sentence doc update. It touches no security-sensitive surface. The code is small and the test exercises hooks, tests, test.each, rejecting and never-settling thenables, and both done() outcomes, and the author's latest commit resolved the only prior inline nit. Defer rather than approve because the PR deliberately chooses to silently ignore a returned thenable's rejection in done tests (1.4.4 behaviour) instead of erroring like Jest, a user-facing test-runner semantics decision a maintainer should confirm; no CODEOWNERS entry covers these paths and no third-party objections are outstanding.

@coderabbitai coderabbitai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Fix comments: Fix CodeRabbit comments on this PR

Comment thread test/js/bun/test/vitest-flavor.test.ts Resolved

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants